Towards Constructivist Assessment ? 1

This paper considers alternative assessment and cybernetics. For more than 30 years, debates about formative and summative assessment have served as surrogates for discussions about the working of the mind, the social implications of assessment and, as important, the role of instruction in the advancement of learning. Arising from these discussions, alternative assessment currently lives uneasily with its classical counterpart. Classical test theory and its conception of the summative value of the true score came from behaviourist learning theories developed in the first three decades of the twentieth century. Formative assessment, with its valued conception of feedback and development, had a different origin. It arose from cognitive and constructivist theories of learning that emerged in the 1930s and 1940s. This paper identifies the tensions that underpin this uneasy coexistence; it suggests that different conceptions of mind lie behind these tensions; and, to mark the autonomy and integrity of formative assessment, it explores an alternative descriptor: ‘constructivist assessment’. The distinction between formative and summative conventionally stems from the work of Michael Scriven, a Australian philosopher of science who studied in Melbourne and Oxford (England). After crossing the Atlantic, Scriven became interested in the application of scientific theory to educational research. By this time (1960–66), he was a professor in the Department of History and Philosophy of Science at Indiana University. In the first of a monograph series on curriculum evaluation, published by the American Educational Research Association, Scriven turned his analytic mind to the fact that ‘current conceptions of the 1 Paper presented at the Annual Conference of the Nordisk Forening for Pedagogisk Forskning (NFPF), Reykjavik, 1014 March, 2004. evaluation of educational instruments...are still inadequate both philosophically and practically’ (1967, p. 39). Building on an earlier paper written while ‘director of the Evaluation Project of the Social Science Education Consortium’ Scriven felt it had become ‘worthwhile’ to distinguish the ‘formative’ and ‘summative’ roles of curriculum evaluation. Scriven’s preference was for summative evaluation performing ‘a final evaluation of the project or person’ (p. 42), but he also acknowledged that colleagues like Lee J. Cronbach who worked at the nearby University of Illinois, prioritised the merits of formative evaluation as ‘part of the process of curriculum development ’ (p. 41). If Cronbach highlighted the formative role of evaluation as something that takes place in the ‘swampy lowland’of curriculum development (Schön,1987, p. 3), Scriven took the high ground offered by summative evaluation. He rebuffed, ‘as a little excessive’, Cronbach’s claim that summative evaluation plays only a ‘menial role’ in curriculum development. Nevertheless, Scriven generously reported Cronbach’s broader view that ‘evaluation, used to improve the course while it is still fluid, contributes more to improvement of education than evaluation used to appraise a product already placed on the market’ (p. 43). In the end, Scriven compromised and took the middle road: that ‘educational projects, particularly curricular ones, clearly must attempt to make best use of evaluation in both these roles’ (p. 43). Further insight into the differences between Scriven and Cronbach can be discerned from the previous article in the same monograph. Written by Robert Gagné (1916-2002), ‘ Curriculum research and the promotion of learning’ offers a particularistic view of learning. It describes curriculum ‘content’ as ‘descriptions of the expected capabilities of students in specified domains of human activity’ (p. 21); and it defines a ‘curriculum’ as ‘a sequence of content units arranged in such a way that the learning of each unit may be accomplished as a single act' (p. 23). In other words, Gagné operated with a view of learning also associated with the name of Burrhus Frederic Skinner (1904 – 1990). Like Gagné, Skinner conceived of learning as the accumulation, by reinforcement, of stimulusresponse associations as Lorrie Shepard reported in a presidential address to the American Educational Research Association: The whole process of becoming competent is any field must be divided into a very large number of very small steps, and reinforcement must be contingent upon the accomplishment of each step. (Skinner 1954, quoted in Shepard, 2000, p.5). Gagné’s views of learning and curriculum also drew on the work of Benjamin Bloom and one of the editors of the monograph series, Ralph Tyler (see, for instance, Tyler 1949; and Bloom et al., 1956). They, too, operated within a behaviouristic framework, emphasising the importance of ‘student behavior’ (Bloom et al. p. 7) and specifications of ‘what students are expected to do’ (Tyler, p. 45). In other words, Gagné’s view of learning built on ideas about learning that dominated American psychology before the 1960s; whereas Cronbach had become aware of the new thinking about learning processes that came to English-language prominence after the 1950s (compare, for instance, Cronbach 1957, and 1975). This historical vignette illustrates an issue central to this paper – that different paradigms for learning imply different paradigms of evaluation, assessment and testing. Accordingly, we believe that the dualism formative/summative should not be represented as two sides of the same thing. Rather, we believe that this coupling is suspect, if not illegitimate. It represents the uneasy coexistence of two different models of evaluation and assessment. The remainder of this paper attempts, first, to identify some of the tensions that underpin this uneasy coexistence; and secondly, to suggest that different conceptions of mind lie behind these tensions. Against this background, we feel that it may be worthwhile to disentangle formative from summative asessment with the help,on the one hand, of a cybernetic theory of feedback and, on the other, the developmental descriptor: ‘constructivist assessment’. Beyond Reflexology During the nineteenth century, research in neuropsychology led to the abandonment of sensory and associationist conceptions of mind. Reflexology, one of the movements that replaced them, was pioneered in the middle of the nineteenth century by Sechenov (Reflexes of the Brain, 1863) and extended through the work of Pavlov in the first part of the 20th century. Within this framework, behaviour is the result of connections between the sensory and motor components of the central nervous system. The brain is regarded as a telephone switchboard or piano keyboard that activates muscular (i.e. behavioural) output. The only question raised by these metaphors was whether each switch or key represents a single muscle or whether they signify more complex stimuli and responses. Accordingly, understanding the relationship between input and output became difficult. What, in fact, happens in the black box of the mind? And, simultaneously, attention to input/output theories raised the general problem of reductionism. If the behaviour of (Pavlov’s) dogs can be explained in terms of reflexology, do the same processes operate in higher animals? Subsequent attention to this problematic feature of human behaviour was taken up in at least three fields of research: communication engineering, physiology and psychology. Communication Engineering and Servo-mechanisms Understanding of feedback reaches back at least as far as the industrial revolution. Although feedback devices had been used for centuries (e.g. in windmills), James Watt’s invention of a centrifugal steam-engine governor in the final years of the eighteenth century is often celebrated as a defining moment. As the rotary speed of the governor declined, a series of operations served to control the steam engine. The valve gradually opened under the force of gravity, more steam was admitted, rotation increased, the revolving balls rose, the flow of steam declined, and the irregularities of the engine were smoothed out. Such feedback, that is, served to steady the working of the machine. New conceptions of feedback arose, however, in the field of electronic engineering. An early example appeared in the American journal Science (1933, July 29th). It was as described as a 'servo-mechanism' – a device that ‘bosses another machine with unusual alertness, speed and accuracy’. Prepared at the instigation of Vannevar Bush, a pioneer of computer science and vicepresident of Massachussetts Institute of Technology (MIT), this research prototype comprised a ‘combination of specially-designed motors, photoelectric cells, vacuum tube amplifiers, and other mechanical devices’. Its task was to follow an ink line drawn on paper, rather like 'a dog with his nose to a trail’. Servomechanisms were analogous to Watt’s centrifugal governor They did not merely keep a machine working steadily, they served to keep it on a pre-planned course. 2 The original article is reproduced at www.sciencenews.org/20030802/timeline.asp During the 1930s, then, the field of control engineering became linked to the field of communication engineering. One of the main actors in this synthesis was Norbert Wiener (18941964), one of Bush’s colleagues at MIT. Communication engineering can be distinguished from control engineering by the fact that its main interest is not ‘economy of energy [Watt’s recurrent problem with the steam engine] but the accurate reproduction of a signal’ (Wiener, 1965, p. 39). Thus control engineering developed new lines of inquiry. It began to place emphasis on the collection, validation and utilisation of information. Cybernetics was the product of this new attention. It is more than the study of how sytems run smoothly or in pre-planned directions; it also includes the notion that systems can steer themselves along new pathways. Wiener took the term cybernetics from the Greek word for a governor and reported his thinking in Cybernetics: or control and communication in the animal and the machine (1965, first edition, 1948) and The Human Uses of Human Beings (1968, first edition 1950). Stated in its simplest form, cybernetics is the study of inputs and outputs. Yet, in both practice and theory, cybernetics had a more complex form. Whenever ‘the simplest control systems are linear', wrote Wiener, 'the output of the effector is a linear expression in the input’ (1965, p. 97). At the same time, however, Wiener recognised that more sophisticated systems – mechanical or human – might manage inputs and outputs in other ways: The information received by the automaton need not be used at once but may be delayed or stored so as to become available at some future time. This is the analogue of memory...[A]s long as the automaton is running, its very rules of operation are susceptible to some change on the basis of the data which have passed through its receptors in the past, and this is not unlike the process of learning. (1965, 43, emphases added) Physiology and Cybernetics As suggested, Wiener’s interest in cybernetics had its roots in mechanical engineering. But his interest in memory had another inspiration – the physiology of nervous systems. He recognised, for example, the significance of memory in the management of feedback: ‘A very important function of the nervous system...is that of memory, the ability to preserve the results of past operations for use in the future’ (1965, p. 121). Further, Wiener’s thinking was not reductionist. He also discussed the difference between lower and higher organisms. Compared with the ‘mechanical rigidity’ of insects, he believed that humans beings display a ‘mechanical fluidity’, especially during their ‘prolonged childhood’. In turn, Wiener suggested that this fluidity had developmental potential, the possibility of ‘almost indefinite intellectual expansion’ (1968 p. 53). From a cybernetic perspective, mankind’s 'mechanical fluidity' is the presupposition of human learning: Feedback is a method of controlling a system by reinserting into it the results of its past performance. If these results are mererly used as numerical data for the criticism of the system and its regulation, we have the simple feedback of the control engineers. If, however, the information which proceeds backwards from the performance is able to change the general method and patterns of performance, we have a process which may well be called learning.(Wiener,1968, p. 56) And it was this fluidity that also underpinned the intersection of brain and engineering research in the 1930s and 1940s. Above all, it helped Wiener and his co-workers to understand the complexity of feedback: While the problem of the conduction of impulses along a fiber may be described in a rather simple way as an all-or-none phenomenon, the problem of the transmission of an impulse across a layer of synaptic connections depends on a complicated pattern of responses, in which certain combinations of incoming fibers, firing within a certain limited time, will cause the message to go further, while certain other combinations will not....These combinations are nto a thing fixed once and for all....They are known to change with temperature, and may well change with many other things. (Wiener, 1968, p. 58) Psychology and Language In distinguishing higher and lower organisms, Wiener also focused on another feature of human communication – language. He foreshadowed, for instance, another MIT colleague Noam Chomsky's critique of B.F. Skinner’s Verbal Behavior. Chomsky argued that input-output or stimulus-response conditioning could not explain any significant aspect of language (Chomsky, 1959). It failed, for instance, to explain how human beings constantly generate sentences that they have never previously uttered. Another linguistic question that attracted Wiener was the problem of semantics. What, in communication engineering, is the relationship between the content of a message and its meaning? Claude Shannon, an early communication theorist and member of Wiener’s intellectual circle wrote 'The semantic aspects of the communication are irrelevant to the engineering problem' (Shannon & Weaver, 1949, p.3; see also Hamilton, Dahlgren, Hult, Roos & Söderström, forthcoming). Wiener, however, thought otherwise: [In] the desire to apply cybernetics to semantics....it seems necessary to make some sort of distinction between information taken brutally and bluntly, and that sort of information on which we as human beings can act effectively or, mutatis mutandis, on which the machine can act effectively. (Wiener,1968, p. 83) Wiener also recognised that the human mind is something more than the human brain: In one sense, this terminal apparatus [i.e. mind] may be regarded as a filter superimposed on the transmission line. Semantically significant information from the cybernetic point of view is that which gets through the line lus filter rather than that which gets through the line alone. In other words, when I hear a passage of music, the greater part of the sound gets to my sense organs and reaches my brain. However, if I lack the perception and training necessary for the aesthetic understanding of musical structure, this information will meet a block, whereas if I were a trained musician it would meet an interpretation structure or organisation which would exhibit the pattern in a significant form which can lead to aesthetic appreciation and further understanding. (Wiener, 1968, p. 84) Although eccentric and idiosyncratic, Wiener’s model for the complexity of mind also served to undermine mechanical accounts of reflexology. Indeed, the brain researcher Oliver Sacks has recently affirmed Wiener’s hesitation. He argues that it has only become possible to 'address such issues as the neural basis of consciousness' with the development of non-invasive scanning techniques developed since Wiener’s death. Further, Sacks affirms Wiener’s non-reductionist view that transmission of a brain impulse is often contingent upon a complicated pattern of intervening responses: A crucial innovation [Sacks writes] has been 'population thinking,' thinking in terms that take account of the brain's huge population of neurons...and the power of experience to differentially alter the strengths of connections between them, and to promote the formation of functional groups or constellations of neurons throughout the brain groups whose interactions serve to categorize experience. (Sacks, 2004, p. 43) Cybernetic thinking in Education To summarise: Wiener’s work is important to this paper in two respects. First, it linked education to the transmission of meaning and the concept of mind: Semantically significant information in the machine as well as in man is information which gets through to an activating mechanism in the system that receives it, despite man’s and/or nature’s attempts to subvert it. From the point of view of Cybernetics, semantics defines the extent of meaning and controls its loss in a communications system. (1968, p. 84) And secondly, Weiner identified a cybernetic link between the transmission of meaning and the steering of goal-related or teleological (i.e. purposive) behaviour. Causality implies a one-way, relatively irreversible functional relationship, whereas teleology is concerned with behaviour not with functional relationships. (Rosenblueth, Wiener & Bigelow; reprinted in Buckley, 1968, p. 225). While feedback has been an element of learning theory since (at least) the time of Sechenov and Pavlov, teleological or cybernetic perspectives on feedback have, as suggested, a more recent provenance. Further, they have animated a connection between feedback and the steering of human activity (cf. Cronbach's preference for formative evaluation). From the perspective of assessment, however, an well-cited source was someone working in the field of systems and management science at Southern Illinois University, Arkalgud Ramaprasad. In an article 'On the definition of feedback', Ramaprasad made the following claim: Feedback is information about the gap between the actual level and the reference level of a system parameter which is use to alter the gap in some way (Ramaprasad, 1983, p. 4). Ramaprasad's position was that feedback is a servo-mechanism for steering a management system. Nevertheless, he adds a further distinction also explicitly linked to communication engineering: Information on the gap when used to alter the gap (most probably to decrease the gap) becomes feedback. If the information on the gap is merely stored without being utilized to alter the gap, it is not feedback (Ramaprasad, 1983, p. 5). One of the first educationist to use Ramprasad’s ideas was, we believe, D. Royce Sadler from the University of Queensland (Australia). In his paper ‘Formative assessment and the design of instructional systems’ (1989), Sadler not only felt that formative assessment ‘requires a distinctive conceptualization and technology’ (p. 120), he gave it a cybernetic focus, even if he did not use the term 'cybernetic': The indispensable conditions for improvement are that the student...is able to monitor continuously the quality of what is being produced during the act of production itself....In other words students have to be able to judge the quality of what they are producing and be able to regulate what they are doing during the doing of it. (Sadler, 1989, p. 121). The discussion of feedback in the early 1990s was also taken up by a Dutch educationist, Marianne Elshout-Mohr, albeit without citing Ramprasad or Sadler. She recognised that the term feedback came 'from cybernetics’, citing Wiener’s Cybernetics; but she also regarded the field as underdeveloped. She was aware, it seems, that cybernetics embraces different conceptions of feedback: What educational psychologists mean precisely by feedback varies. One can simply understand it to mean "the obtaining of information of good (corresponding to a norm) or bad (diverging from a norm) quality of what the pupils produce (or reproduce)”. Or it can be defined more broadly. [Where] The term feedback is used more to indicate a educational function that should be fulfilled in all learning processes, even the higher ones. (Elshout-Mohr, 1994, p. 59). Despite this diversity, Elshout-Mohr also appreciated that a close link could be forged between cybernetics and education. To do this she drew on a Dutch dissertation published in 1990: Feedback is the procurement of information on a process (one wishes to master) by making measurements at the output of a system in which the process is taken place, then comparing the measurement results with norms and, if there are divergences from a defined tolerance, intervening in the process in such a way that the aim of the system in which the process takes place is achieved (Roossink, quoted in Elshout-Mohr, 1994, p. 59). Like Ramprasad, Elshout-Mohr's view of feedback relates, perhaps, more to servo-mechanics than cybernetics. Later, however, Sadler made a further analysis of feedback, revealing his understanding of its historical origins as well as its cybernetic substance: Perhaps the first phase [of feedback thinking] flowed from stimulus response theory and identified feedback with knowledge of results. This could then lead to remediation (when incorrect) or reinforcement (when correct), higher motivation and hence higher achievement. The second phase took a different tack: feedback was concerned with praise for effort, which would lead to higher self-esteem, more effort, and finally higher achievement. Recent research pays more attention to specific feedback tailored to both the nature of the assessment task and the learner’s response to that task, progressive appreciation by the learner of what constitutes high quality work and the strategies needed to attain high standards, and hence high achievement. (1998, p. 78). Moreover, Sadler also echoed Wiener's earlier insight that feedback is part of communication engineering: “If the information is simply recorded ... or is too deeply coded (for example, as a summary grade given by the teacher) to lead to appropriate action, the control loop cannot be closed and ‘dangling data’ substitute for effective feedback” (Sadler, 1989, p. 121). Such summative assessments might be formative in purpose but they would not be formative in function since their relevance to the human activities of self-steering and self-direction is undefined. Feedback and Constructivism The key insight of Wiener, progressively recapitulated by Ramprasad, Elshout-Mohr and Sadler, is that cybernetic feedback should have structural and goal-directed consequences for the organisation of learning. This teleological or cybernetic insight, which was launched in the classic paper on cybernetics ( Rosenblueth, Wiener, & Bigelow, 1943), is carefully restated in an introduction to a reprint of the original paper: ‘self-regulation’ and ‘self-direction’... are not to be assumed to be near synonyms, but are intended to point up a central distinction between two general kinds of processes that may be found in complex systems. Some mechanisms are essentially conserving....Homeostatis is the classic example. On the other hand, there is another class of processes whose effect is to direct not only the system's behaviour but also the very nature of its structure or organisation in new, often 'higher' directions that is, to change or elaborate its given structure into a significantly different one. (Buckley, 1968, p. 219) Cybernetic attention to 'mind' was part of the (US) cognitive revolution launched in 1956 (see, for instance, Bruner, 1983, p. 120-121). But, from Wiener's cybernetic perspective, the cognitive revolution also included a constructivist or ‘self-direction’ outcome. That is, feedback not only plays a part in the ‘self-regulation’ (or homeostasis) of human learning, it can also foster human development. In short, feedback can have a constructivist dimension, based on the asssumption that human thought can operate at different levels, each one of a ‘logical type’ (Wiener, 1968, p. 545, using a formulation taken from another mathematician, Bertrand Russell). The work of Jean Piaget (1896-1960) provides an illustration of the new kind of cognitivist and constructivist thinking that penetrated, from Europe, into Anglo-American thought. As a professor of psychology at the university of Geneva, Piaget's pivotal ideas about mind were threefold. He embraced the 'idea of wholeness, the idea of transformation, and the idea of self-regulation' (Piaget, 1971, p. 5). The idea of wholeness is analogous to the idea that the elements of a system have relational properties that hold them together. The idea of transformation arises whenever elements of a system interact to create new relationships. And the idea of self-regulation is associated with the idea that transformation arises from the interaction of the elements or the relational properties of a system. Piaget took these ideas into his analysis of the 'structure and the genesis of intelligence' (Piaget, 1971, p. 60ff). Above all, he linked the notion of self-regulation to self-construction. The representation of human mental structure includes 'general co-ordinations of actions' (p. 62) which, through a process of assimilation and accommodation or self-regulation and transformation generate new forms of human intelligence. In the process: The subject's activity calls for a continual 'de-centering' without which he cannot become free from his spontaneous intellectual egocentricity. This 'de-centering' makes the subject enter upon, not so much an already available and therefore external universality, as an uninterrupted process of coordination and setting in reciprocal relations....The subject exists because, to put it very briefly, the being of structures consists in their coming to be, that is, their being 'under construction'. (Piaget, 1971, pp. 139-140) Indeed, this view of knowledge allowed Piaget to conclude that: 'There is no structure apart from construction’ (p. 140). Piaget is not the only source of constructivist ideas, even if he was one of the first to inject ideas about 'cybernet structuralism' into educational thought (p. 134). Subsequently, constructivism has become a diverse field of action and inquiry. Nevertheless, a small number of assumption account for its integrity. First, it assumes that individuals are inducted into ways of thinking, working and seeing that, in turn, can be described as qualitatively different. Secondly, constructivism assumes that the acquisition of learning entails an incremental or step-wise process of development. Finally, constructivism assumes that learned abilities are fostered through mediated learning opportunities With the help of the scaffolding implicated in their language and environment, learners acquire the capacity to transcend their experience. Here are four statements that can be interpreted in terms of constructivist development: a. this is a big one b. this one is bigger than that one c. The difference between these sizes is geometric not arithmetic because each one is twice as big as the adjacent one. d. I understand different types of measurement scale While the first example illustrates Piaget's 'general coordinatons of actions', The remaining examples illustrate, in Shepard’s terms (2001, p. 1076), ‘true understanding’. They indicate selfmonitoring (Shepard’s formulation) and the potential for self-direction. That is, they comprise the capacity to evaluate, build upon and, ultimately, transcend prior knowledge. Indeed, Black and Wiliam echoing Sadler have made the same constructivist judgement about the wholeness of instruction, the transformation of learning and the self-regulation of performance: ‘more will be gained from formative feedback where a test calls for the mindfulness that it helps to develop’ (Black & Wiliam, 2003, p. 631, emphasis added). Constructivist Assessment Cybernetic feedback, therefore, is at the heart of constructivist monitoring. It operates within the learner, between learners or between a teacher and a learner. Assessment is therefore seen as part of the mediation of teaching and learning. Monitoring is mutual; and the resultant exchanges foster human development. Further, cybernetic feedback implies the establishment of new kinds of pedagogic relationships – learning contracts or environments between teachers and learners. Besides placing ‘mindfulness’ at the centre of their own efforts to develop formative assessment, Black and Wiliam also identified the importance of ‘supported development’ for teachers; and the teaching and learning potential of ‘rich questions, comment-only marking, sharing criteria with learners, and student peerand selfassessment’ (2003, p. 630). When asked ‘what makes for good feedback?’, Black and Wiliam evolved a simple cybernetic answer ‘good feedback causes thinking’ (p. 631). Conclusion This paper attempts to provide formative assessment with its own autonomy and integrity. At root it builds on two ideas. First, that the distinction between formative and summative is rooted in fundamentally different conceptions of learning. And secondly, that these conceptions of learning entail distinct conceptions of mind. To identify these differences, the paper attends to the learning theories of the last 100 years and, in particular, to the diverse insights that accompanied the digitalisation of comunication engineering. The general argument of this paper is that feedback can take different forms and that an understanding of these forms is relevant not only to educational assessment but also to wider educational issues about fostering of human development. Finally, our discussion of feedback may also contribute to the clarification of the constructivist concept of 'scaffolding'. We started this article with aim of distinguishing formative from summative. We felt that it may not be helpful to treat them as opposite sides of the same thing. Accordingly, we envisage formative assessment/evaluation, from Wiener’s perspective, as an educational process which seeks to control the ‘loss of meaning’ in educational communication; whereas, from Sadler’s perspective, we see summative assessment/evaluation as a procedure that is ‘too deeply coded’ to become, in Wiener's terms, a valid ‘activating mechanism’ for goal-directed educational activities. Finally, we recognise that this claim is analytically controversial and practically complex. Where, in practice, is the boundary between formative and summative assessment/evaluation? Where is the boundary between self-regulation and self-direction? And where is the boundary between rich and poor questioning? This paper cannot answer these questions. But we hope that it might be an 'activating mechanism' for others looking for further insight into formative and summative activities.